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LONG-TERM  GOALS 

The  long-term  goal  is  to  develop  a  parallel  ocean  simulation  tool  that  is  capable  of  simulating 
processes  on  a  wide  range  of  scales  by  coupling  two  vastly  different  codes,  namely  the  Regional  Ocean 
Modeling  System  (ROMS,  Shchepetkin  and  McWilliams  (2005)),  and  the  Stanford  Unstructured 
Nonhydrostatic  Terrain- following  Adaptive  Navier-Stokes  Simulator  (SUNTANS,  Fringer  et  al. 
(2006)).  The  tool  will  adaptively  nest  SUNTANS,  an  unstructured-grid,  coastal-scale  code,  into 
ROMS,  a  curvilinear  grid,  regional-scale  code,  in  regions  where  the  motions  are  small-scale  and  so 
nonhydrostatic.  The  nested  tool  will  be  applied  to  study  highly  nonlinear  internal  waves  in  the  South 
China  Sea  in  order  to  develop  an  improved  understanding  of  mechanisms  that  govern  their  generation, 
propagation,  and  dissipation. 

OBJECTIVES 

The  primary  objective  is  to  enhance  the  capabilities  of  the  SUNTANS  model  through  development  of 
algorithms  to  study  multiscale  processes  in  estuaries  and  the  coastal  ocean.  This  involves  development 
of  1)  improved  momentum  and  scalar  advection  on  unstructured,  staggered  grids,  2)  accurate  and 
efficient  algorithms  for  solution  of  the  nonhydrostatic  pressure,  and  3)  adaptive  grid  capabiliites  with 
adaptive  mesh  refinement  and  model  nesting. 

APPROACH 

This  work  focuses  on  the  continued  development  of  SUNTANS  (Stanford  Unstructured 
Nonhydrostatic  Terrain- following  Adaptive  Navier-Stokes  Simulator),  a  free-surface,  nonhydrostatic, 
unstructured-grid,  parallel  coastal  ocean  and  estuary  simulation  tool  that  solves  the  Navier-Stokes 
equations  under  the  Boussinesq  approximation  (Fringer  et  al.,2006).  The  formulation  is  based  on  the 
method  outlined  by  Casulli  and  Walters  (2000),  in  which  the  free-surface  and  vertical  diffusion  are 
discretized  with  the  theta  method,  which  eliminates  the  Courant  condition  associated  with  fast  free- 
surface  waves  and  the  elevated  friction  tenn  associated  with  small  vertical  grid  spacing  at  the  free- 
surface  and  bottom  boundary.  For  flows  with  extensive  wetting  and  drying,  advection  of  momentum  is 
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accomplished  with  the  semi-Lagrangian  advection  scheme  (Wang  et  al.  2011a),  which  ensures  stability 
in  the  presence  of  cells  that  fill  and  empty  with  the  tides.  Scalar  advection  is  accomplished  semi- 
implicitly  and  continuity  of  volume  and  mass  are  guaranteed  for  the  hydrostatic  solver,  following  Gross 
et  al.  (2002).  The  theta  method  for  the  free-surface  yields  a  two-dimensional  Poisson  equation,  and  the 
nonhydrostatic  pressure  is  governed  by  a  three-dimensional  Poisson  equation.  These  are  both  solved 
with  the  preconditioned  conjugate  gradient  algorithm  with  Jacobi  and  block-Jacobi  preconditioning, 
respectively.  Because  the  nonhydrostatic  component  of  SUNTANS  is  essentially  a  correction  to  the 
hydrostatic  component,  SUNTANS  can  be  run  seamlessly  in  nonhydrostatic  or  hydrostatic  modes. 
SUNTANS  is  written  in  the  C  programming  language,  and  the  message-passing  interface  (MPI)  is 
employed  for  use  in  a  distributed-memory  parallel  computing  environment.  SUNTANS  employs  the 
generalized  length  scale  approach  to  Reynolds-averaged  turbulence  modeling  (Wang  et  al.  2011b). 

The  SUNTANS  grid  employs  z-levels  in  the  vertical  and  is  unstructured  in  plan,  which  enables  the 
resolution  of  complex  coastlines  and  topographic  features.  Unstructured  grids  also  enable  the  use  of 
high  grid  resolution  in  regions  of  interest  while  coarsening  the  grid  in  regions  where  grid  resolution  is 
not  required,  thereby  significantly  reducing  computational  overhead. 

WORK  COMPLETED 

We  have  developed  a  criterion  to  determine  grid  resolution  requirements  to  resolve  nonhydrostatic 
internal  waves.  This  is  important  for  developing  strategies  for  adaptive  mesh  refinement  for 
nonhydrostatic  processes. 

RESULTS 

Vitousek  and  Fringer  (2011)  show  that  nonhydrostatic  dispersion  is  the  basic  physical  feature  that 
differentiates  hydrostatic  from  nonhydrostatic  models.  Dispersion  is  essential  for  the  simulation  of 
solitary-like  waves  because  it  balances  nonlinear  steepening,  the  process  responsible  for  the  wave 
speed  increasing  with  wave  height.  Although  hydrostatic  models  do  not  possess  physical  dispersion, 
most  ocean  models  are  second-order  accurate  in  space,  and  so  they  possess  numerical  dispersion  which 
can  mimic  the  physical  effects  of  nonhydrostatic  dispersion.  Vitousek  and  Fringer  have  demonstrated 
that,  contrary  to  common  belief,  it  is  not  the  advection  tenns  but  rather  the  pressure  gradient  tenns  that 
are  the  dominant  cause  of  numerical  dispersion  in  internal  wave  simulations.  In  nonhydrostatic 
models,  accurate  simulation  of  solitary  waves  requires  that  the  grid  resolution  be  fine  enough  to  ensure 
that  the  numerical  dispersion  is  much  smaller  than  the  physical  nonhydro  static  dispersion. 

Vitousek  and  Fringer  (2011)  show  for  accurate  simulation  that  this  leads  to  the  constraint  X=  Ax/hi<l, 
where  X  is  the  grid  lepticity  (or  "thinness",  from  Scotti  and  Mitran  2009),  Ax  is  the  horizontal  grid 
spacing,  and  hi  is  the  depth  of  the  wave  guide.  Figures  1  and  2  compare  the  result  of  an  evolving 
solitary-like  wave  when  computed  with  SUNTANS,  in  both  hydrostatic  and  nonhydrostatic  modes.  All 
conditions  are  the  same  except  for  X.  As  shown  in  Figure  1,  for  large  values  of  A,,  nonhydrostatic 
effects  are  not  resolved  and  the  hydrostatic  and  nonhydrostatic  results  are  essentially  identical.  Figure 
1  also  shows  how  the  hydrostatic  model  produces  solitary  waves,  but  this  is  purely  a  numerical  effect 
since  the  nonlinear  steepening  is  in  balance  with  the  numerical  dispersion.  As  shown  in  Figure  2,  for 
small  values  of  X,  nonhydrostatic  effects  are  now  resolved  and  there  are  significant  differences  between 
the  hydrostatic  and  nonhydrostatic  model  results  with  a  well  defined  and  realistic  wave  train  evolving 
in  the  nonhydrostatic  case. 
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Figure  1:  Comparison  of  the  evolution  of  a  solitary-like  wave  using  hydrostatic  SUNTANS  (red) 
and  nonhydrostatic  SUNTANS  (blue)  with  A=  Ax/h]=8  (from  Vitousek  and  Fringer  2011). 
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Figure  2:  Comparison  of  the  evolution  of  a  solitary-like  wave  using  hydrostatic  SUNTANS  (red) 
with  nonhydrostatic  SUNTANS  (blue)  with  A=  Ax/hi=0.25  (from  Vitousek  and  Fringer  2011). 


3 


The  criterion  of  Vitousek  and  Fringer  (2011)  has  important  ramifications  for  nonhydrostatic  modeling 
of  internal  waves.  Consider,  for  example,  numerical  simulation  of  nonhydrostatic  waves  in  a  coastal 
shelf  domain.  Since  the  typical  mixed-layer  depth  is  0(10  m),  then  the  horizontal  grid  resolution  must 
be  less  than  10  m,  implying  that  billions  of  grid  cells  are  needed  to  simulate  internal  waves  in  a  100  km 
coastal  shelf  domain.  With  modem  computing  resources,  simulations  with  100  million  grid  cells 
require  100s  of  processors  running  for  weeks  at  a  time  to  simulate  a  month  of  real  time.  A  coarser  grid 
would  not  sufficiently  resolve  the  physical  nonhydrostatic  dispersion  and  the  resulting  waves  would  be 
numerical  solitons.  Depending  on  the  resolution,  computation  of  the  nonhydrostatic  pressure  may  even 
degrade  the  results  because  the  physical  nonhydrostatic  dispersion  would  be  in  addition  to  the 
(incorrect)  numerical  dispersion.  As  an  example,  Figure  3  compares  internal  waves  in  the  South  China 
Sea  (Zhang  et  al.  2011)  computed  with  and  without  the  nonhydrostatic  pressure.  These  simulations 
employ  a  horizontal  grid  resolution  of  1400  m.  With  a  mixed-layer  depth  of  200  m,  this  implies  a 
lepticity  of  Z=(1400  m)/(200  m)=7.  Therefore,  the  numerical  dispersion  overwhelms  the  physical 
dispersion  by  a  factor  of  3.6  (following  Vitousek  and  Fringer  2011),  and  so  the  nonhydrostatic  result  in 
Figure  3  is  overly  dispersive  because  it  includes  the  effects  of  both  physical  and  numerical  dispersion. 


Figure  3:  Free-surface  displacement  (in  m  above  mean  sea  level)  and  the  associated  internal  wave 
signatures  at  12:30am  on  25  June  2005  around  Dongsha  Atoll  as  computed  by  the  hydrostatic  (left) 
and  nonhydrostatic  (right)  versions  of  SUNTANS  (from  Zhang  et  al.  2011). 

The  grid-resolution  criterion  of  Vitousek  and  Fringer  (2011)  imposes  extensive  computational 
requirements  on  numerical  models  of  internal  waves  in  coastal  domains.  The  computational  overhead 
is  large  not  only  because  of  the  number  of  grid  points  required,  but  also  because  of  the  overhead  arising 
from  the  three-dimensional  elliptic  equation  associated  with  the  nonhydrostatic  pressure.  Fortunately, 
however,  as  shown  in  Figure  4,  computational  overhead  associated  with  the  nonhydrostatic  pressure 
solver  is  only  high  when  the  grid  leptic  ratio  is  low.  This  behavior  of  the  computational  overhead 
arises  with  use  of  suitable  preconditioners  for  the  nonhydrostatic  pressure  solver  (Fringer  et  al.  2006) 
that  eliminate  computational  overhead  when  nonhydrostatic  processes  are  not  being  resolved,  or  when 
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A>  I .  Therefore,  unstructured-grid  models  like  SUNTANS  can  be  designed  to  employ  low-lepticity 
grids  only  where  physical  nonhydrostatic  dispersion  is  expected,  such  as  on  continental  shelves.  The 
computational  overhead  associated  with  the  nonhydro static  pressure  can  thus  be  incurred  only  where  it 
is  necessary. 


Figure  4:  Relative  workload  associated  with  computing  the  nonhydrostatic  pressure  as  a  function  of 
the  grid  lepticity  A  =  Ax/hi .  The  overhead  is  computed  as  the  ratio  of  the  total  wallclock  time  for  the 
nonhydrostatic  simulation  to  that  of  the  hydrostatic  simulation  to  obtain  the  results  shown  in 
Figures  1  and  2  (from  Vitousek  and  Fringer  2011). 

IMPACT/APPLICATIONS 


High-resolution  simulations  using  nonhydro  static  models  like  SUNTANS  are  crucial  for  understanding 
multiscale  processes  that  are  unresolved,  and  hence  parameterized,  in  larger-scale  ocean  models. 
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